With continuing development of new and better ways of delivering television and other video presentations to end users, and parallel development of computerized information systems, such as the Internet and the associated World Wide Web (WWW), there have been concerted efforts to integrate various systems to provide enhanced information delivery and entertainment systems. For example, developers are introducing integrated systems combining TVs with computer subsystems, so a TV may be used as a WEB browser, or a PC may be used for enhanced TV viewing.
In some systems computer elements, such as a CPU, memory, and the like, are built into the familiar chassis of a TV set. In such a system, the TV screen becomes the display monitor in the computer mode. In such a system, conventional TV elements and circuitry are incorporated along with the computer elements, and capability is provided for a user to switch modes, or to view recorded or broadcast video with added computer interaction. One may thus, with a properly equipped system, select to view analog TV programs, digital TV programs, conventional cable TV, satellite TV, pay TV from various sources, and browse the WWW as well, displaying WEB pages and interacting with on-screen fields and relational systems for jumping to related information, databases, and other WEB pages. The capabilities are often integrated into a single display, that is, one may view a broadcast presentation and also have a window on the display for WEB interaction.
In some other systems, computer elements are provided in an enclosure separate from the TV, often referred to in the art as a set-top box. Set-top box systems have an advantage for providers in that they may be connected to conventional television sets, so end users don't have to buy a new TV along with the computer elements.
In such integrated systems, whether in a single enclosure or as set-top box systems, user input is typically through a hand-held device quite similar to a familiar remote controller, usually having infra-red communication with the set-top box or a receiver in the integrated TV. For computer modes, such as WEB browsing, a cursor is displayed on the TV screen, and cursor manipulation is provided by buttons or other familiar pointer apparatus on the remote. Select buttons are also provided in the remote to perform the familiar function of such buttons on a pointer device, like a mouse or trackball more familiar to computer users.
Set-top boxes and computer-integrated TVs adapted as described above typically have inputs for such as a TV antenna (analog), cable TV (analog or digital), more recently direct-satellite TV (digital), and may also connect to video cassette recorders and to mass storage devices such as hard disk drives and CD-ROM drives to provide a capability for uploading video data from such devices and presenting the dynamic result as a display on the TV screen.
The inventors note that the innovations and developments described above provide enhanced ability to view and interact with video presentations, and that the quality of presentation and efficiency of interaction will be at least partly a function of the computer power provided and the sophistication and range of the hardware and software.
The present inventors have noted that even with the advances in hardware and software so far introduced in the art, there is still considerable room for improvement, and the inventors have accordingly provided a unique system for processing video streams, determining topic changes, and marking the streams at topic change points with thumbnails, in some cases annotated. The system is particularly useful for preparing and presenting media-rich interactive video presentations termed I-Mag by the inventors Digital content presented in the interactive magazine taught by the co-related patent specification listed in the cross-reference section is generated in many instances from broadcast analog content that is converted to digital video during off-line authoring processes. Interactive thumbnails representing entry points to new video content offered in the video magazine are generated using scene-change-detection technologies (SCD) and presentation time stamp (PTS) technologies, both of which are known in the art and to the inventor. SCD uses significant color changes to overall color levels from frame to frame to determine when a new video segment or a significant story change has occurred in a video presentation. In this way, thumbnail pictures may be presented in a user-interface along with the video that is currently playing such that a user may interact with the thumbnails to jump to the represented portion of the video presentation or obtain additional information related to that section of the magazine or video segment.
In combination with SCD software, an off-line video editor must manually group and sort such thumbnail pictures for presentation in the interactive magazine. In many cases, an editor will view a presentation off-line while performing editing processes using automated as well as manual software processes to accomplish the task of completing an interactive magazine that is ready for download to users interacting with a central WEB-based server. Such off-line processing can be time consuming and can, at times, command considerable resources both human and machine.
It has occurred to the inventors that the time and resource dedicated to off-line authoring of raw video content that will eventually be included, for example, in an interactive video magazine, or for any other use, may be considerably reduced through automated processing. This requires that a more exact method than SCD be used for determining where content and topic changes occur in a video presentation. SCD technology, while very helpful, remains a non-exact procedure for determining scene changes requiring human supervision in order to correct mistakes made by the software. Moreover, success of SCD techniques may rely heavily on the type and format of raw content to be authored.
Interactive video content presented in the interactive magazine as known to the inventors is derived largely from analog video content that has been previously broadcast to the public such as news programs, movies made for TV, educational programs, and so on. Many of these programs are segmented such as a news program wherein several different stories are covered before going to a commercial break. It becomes difficult for an editor using SCD technology using color change detection to determine, for example, the point at which one story ends and another begins in a news cast.
Therefore, what is clearly needed is a method and apparatus that can be used to automatically and accurately detect when a topic or a story line changes in a raw video segment during off-line authoring of such content. Such a method would allow a video editor to concentrate on other editing tasks without being required to supervise the scene change detection process.